{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 第十一章 特征工程之数据预处理（11.6至11.7）"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 11.6 多重共线性的分析与处理"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>X1</th>\n",
       "      <th>X2</th>\n",
       "      <th>X3</th>\n",
       "      <th>Y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>8</td>\n",
       "      <td>16</td>\n",
       "      <td>-32</td>\n",
       "      <td>77</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>7</td>\n",
       "      <td>14</td>\n",
       "      <td>-31</td>\n",
       "      <td>52</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>4</td>\n",
       "      <td>9</td>\n",
       "      <td>-12</td>\n",
       "      <td>42</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>8</td>\n",
       "      <td>19</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   X1  X2  X3   Y\n",
       "0   8  16 -32  77\n",
       "1   7  14 -31  52\n",
       "2   4   9 -12  42\n",
       "3   1   2   5   6\n",
       "4   1   2   8  19"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 使用Pandas库读入一组存在多重共线性的数据，并对其回归作为示例：\n",
    "import pandas as pd\n",
    "df = pd.read_excel('数据.xlsx')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 对数据集划分特征变量和目标变量：\n",
    "X = df.drop(columns='Y')\n",
    "Y = df['Y']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面我们需要做的就是分析与检验这三个特征变量是否存在多重共线性，这里主要讲解两种判别方法：相关系数判断以及方差膨胀因子法（VIF检验）来检验多重共线性。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1.相关系数判断"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>X1</th>\n",
       "      <th>X2</th>\n",
       "      <th>X3</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>X1</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.992956</td>\n",
       "      <td>-0.422788</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>X2</th>\n",
       "      <td>0.992956</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>-0.410412</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>X3</th>\n",
       "      <td>-0.422788</td>\n",
       "      <td>-0.410412</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "          X1        X2        X3\n",
       "X1  1.000000  0.992956 -0.422788\n",
       "X2  0.992956  1.000000 -0.410412\n",
       "X3 -0.422788 -0.410412  1.000000"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X.corr()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其中第i行第j列的内容表示的就是对应的第i个特征变量和第j个特征变量的相关系数，例如第1行第2列的相关系数：0.99表示的就是特征变量X1和特征变量X2的相关系数，可以看到这两个特征变量的相关性还是非常强的，因此有理由相信这两个变量会导致多重共线性的现象，因此需要删去其中一个特征变量。注意对角线上的相关系数都为1，这个其实没有什么意义，因为它表示的是自身与自身的相关系数，那自然是1了。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2.方差膨胀因子法（VIF检验）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 为了检验上述回归中是否存在严重的多重共线性，我们使用Python的VIF检验模块来验证：\n",
    "from statsmodels.stats.outliers_influence import variance_inflation_factor\n",
    "vif = [variance_inflation_factor(X.values, X.columns.get_loc(i)) for i in X.columns]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[259.6430487184967, 257.6315718292196, 1.302330632715429]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vif"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[259.6430487184967, 257.6315718292196, 1.302330632715429]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 如果对上面的快捷写法不太理解，上面的代码也可以写成：\n",
    "vif = []\n",
    "for i in X.columns:  # i对应的是每一列的列名\n",
    "    vif.append(variance_inflation_factor(X.values, X.columns.get_loc(i)))\n",
    "    \n",
    "vif"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 对数据集重新划分特征变量和目标变量：\n",
    "X = df[['X1', 'X3']]\n",
    "Y = df['Y']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[1.289349054516766, 1.289349054516766]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 进行检验VIF检验：\n",
    "from statsmodels.stats.outliers_influence import variance_inflation_factor\n",
    "vif = [variance_inflation_factor(X.values, X.columns.get_loc(i)) for i in X.columns]\n",
    "\n",
    "vif"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 11.7 过采样和欠采样"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "11.7.1 过采样"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>编号</th>\n",
       "      <th>年龄</th>\n",
       "      <th>负债比率</th>\n",
       "      <th>月收入</th>\n",
       "      <th>贷款数量</th>\n",
       "      <th>家属人数</th>\n",
       "      <th>分类</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>29</td>\n",
       "      <td>0.22</td>\n",
       "      <td>7800</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>52</td>\n",
       "      <td>0.46</td>\n",
       "      <td>4650</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>28</td>\n",
       "      <td>0.10</td>\n",
       "      <td>3000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>29</td>\n",
       "      <td>0.20</td>\n",
       "      <td>5916</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>27</td>\n",
       "      <td>1.28</td>\n",
       "      <td>1300</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   编号  年龄  负债比率   月收入  贷款数量  家属人数  分类\n",
       "0   1  29  0.22  7800     1     3   0\n",
       "1   2  52  0.46  4650     1     0   0\n",
       "2   3  28  0.10  3000     0     0   0\n",
       "3   4  29  0.20  5916     0     0   0\n",
       "4   5  27  1.28  1300     0     0   1"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "data = pd.read_excel(\"信用卡数据.xlsx\")\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 通过如下代码提取特征变量并将其赋值给变量X，提取目标变量并将其赋值给变量y：。\n",
    "X = data.drop(columns='分类')\n",
    "y = data['分类']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Counter({0: 1000, 1: 100})"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 然后使用collections库中的Counter()方法，对目标变量进行计数：\n",
    "from collections import Counter\n",
    "Counter(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "不违约的样本数有1000个，远远大于违约的样本数100。为了防止建立信用违约模型时，模型着重拟合不违约的样本，而无法找出违约的样本，我们采用过采样的方法来改善样本比例不均衡的问题，这里我们将通过上面讲到的随机过采样和SMOTE法过采样来进行代码实现。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "（1）随机过采样"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "from imblearn.over_sampling import RandomOverSampler\n",
    "ros = RandomOverSampler(random_state=0)\n",
    "X_oversampled, y_oversampled = ros.fit_resample(X, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Counter({0: 1000, 1: 1000})"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 用Counter()方法检验一下SMOTE过采样的效果。\n",
    "Counter(y_oversampled)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "违约的样本数从100上升至不违约的样本数1000，这证明我们的随机过采样有效。同时我们可以打印特征变量X_oversampled的shape来看看特征变量的变化："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2000, 6)"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_oversampled.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这里的2000就是1000个违约样本和1000个不违约样本相加得到的，可以看到，随机过采样后特征变量的数据也随之增多。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "（2）SMOTE过采样"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 通过如下代码即可进行SMOTE过采样：\n",
    "from imblearn.over_sampling import SMOTE\n",
    "smote = SMOTE(random_state=0)\n",
    "X_smotesampled, y_smotesampled = smote.fit_resample(X, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Counter({0: 1000, 1: 1000})"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 用Counter()方法检验一下SMOTE过采样的效果。\n",
    "Counter(y_smotesampled)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "11.7.2 欠采样"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 仍然采用上方的信用违约数据进行欠采样代码的展示：\n",
    "from imblearn.under_sampling import RandomUnderSampler\n",
    "rus = RandomUnderSampler(random_state=0)\n",
    "X_undersampled, y_undersampled = rus.fit_resample(X, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Counter({0: 100, 1: 100})"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 用Counter()方法检验一下随机欠采样的效果。\n",
    "Counter(y_undersampled)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(200, 6)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 不违约的样本数从1000下降至违约的样本数100，这证明我们的随机欠采样有效。同时我们可以打印特征变量X_undersampled的shape来看看特征变量的变化：\n",
    "X_undersampled.shape"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
